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Abstract. We discuss stochastic dynamics of finite populations of individuals playing games. 
We review recent results concerning the dependence of the long-run behavior of such systems 
on the number of players and the noise level. In the case of two-player games with two symmet- 
ric Nash equilibria, when the number of players increases, the population undergoes multiple 
transitions between its equilibria. 
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1 Introduction 



Many socio-economic and biological processes can be modeled as systems of interacting in- 
dividuals; see for example Santa Fe collection of papers on economic complex systems [1], 
econophysics bulletin [2], and statistical mechanics and quantitative biology archives [3]. 

Here we will consider game-theoretic models of many interacting agents [4, 5, 6]. In such 
models, agents have at their disposal certain strategies and their payoffs in a game depend 
on strategies chosen both by them and by their opponents. A configuration of a system, 
that is an assignment of strategies to agents, is a Nash equilibrium if for any agent, for fixed 
strategies of his opponents, changing the current strategy will not increase his payoff. One of the 
fundamental problems in game theory is the equilibrium selection in games with multiple Nash 
equilibria. In two-player games with two strategies we may have two Nash equilibria: a payoff 
dominant (also called efficient) and a risk-dominant one. In the efficient equilibrium, players 
receive highest possible payoffs. The strategy is risk-dominant if it has a higher expected payoff 
against a player playing both strategies with equal probabilities. It is played by individuals 
averse to risks. 

One of the selection methods is to construct a dynamical system where in the long run only 
one equilibrium is played with a high frequency. Here we will discuss an adaptive dynamics 
introduced by Robson and Vcga-Rcdondo [7]. In their model, at any time period, individuals 
play only one game with randomly chosen opponents (they do not play against an average 
strategy as in the replicator dynamics or the adaptive model of Kandori, Mailath, and Rob 
[8]). The selection part of the dynamics ensures that if the mean payoff of a given strategy at 
the time t is bigger than the mean payoff of the other one, then the number of individuals playing 
the given strategy should increase in t + 1. In addition, with a small probability representing 
the noise of the system, players may make mistakes. 

To describe the long-run behavior of stochastic dynamics, Foster and Young [9] introduced 
a concept of stochastic stability. A state of a system (a number of individuals playing the first 
strategy in our models) is stochastically stable if it has a positive probability in the stationary 
state in the limit of zero noise. It means that in the long run we observe it with a positive 
frequency. 

Here we review recent results concerning the dependence of the long-run behavior of the 
above desribed dynamics on the number of players and the noise level. We will combine these 
results to show that in the case of two-player games with two symmetric Nash equilibria, when 
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the number of players increases, the population undergoes multiple transitions between its 
equilibria. 

2 Adaptive dynamics with mistakes 

We will consider a finite population of n individuals who have at their disposal one of two 
strategies: A and B. At every discrete moment of time, t — 1, 2, they are randomly paired 
(we assume that n is even) to play a two-player symmetric game with payoffs given by the 
following matrix: 

A B 
A a b 

U = 

Bed 

where the ij entry, i,j = A,B, is the payoff of the first (row) player when he plays the 
strategy i and the second (column) player plays the strategy j. We assume that both players 
are the same and hence payoffs of the column player are given by the matrix transposed to U ; 
such games are called symmetric. 

An assignment of strategies to both players is a Nash equilibrium, if for each player, for a 
fixed strategy of his opponent, changing the current strategy will not increase his payoff. If 
a > c and d > h, then {A, A) and {B, B) are two Nash equihbria. If a + 6 < c + c?, then the 
strategy B has a higher expected payofi' against a player playing both strategies with equal 
probabilities. We say that B risk dominates the strategy A (the notion of the risk-dominance 
was introduced and thoroughly studied by Harsanyi and Selten [10]). If in addition a > d, 
then we have a selection problem of choosing between the payoff-dominant (also caled efficient) 
equilibrium {A, A) and the risk-dominant {B,B). 

At every discrete moment of time t, the state of our population is described by the number 
of individuals, Zt, playing A. Formally, by the state space we mean the set 

fl = {z,0 < z <n}. 

Now we describe the dynamics of our system. It consists of two components: selection and 
mutation. The selection mechanism ensures that if the mean payoff of a given strategy, 
7ri{zt),i = A,B, at the time t is bigger than the mean payoff of the other one, then the 
number of individuals playing the given strategy should increase in t -|- 1. 
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Let pt denote the random variable which describes the number of cross-pairings, i.e. the 
number of pairs of matched individuals playing different strategies at the time t. Let us notice 
that Pt depends on Zf. For a given realization of pt and Zt, mean payoffs obtained by each 
strategy are as follows: 

a{zt-pt) + bpt 

7^A{zt,Pt) = , (1) 

. cpt + d{n -Zt- Pt) 

T^B{zt,Pt) = , 

n- Zt 

provided < Zt < n. 

We assume that at any time period, each individual has a revision opportunity with a small 
positive probability r and adopts a strategy with the higher mean payoff. Players may make 
mistakes. At every time period, each player who has a revision opportunity, instead of following 
the selection rule may adopt the other strategy with a small probability e. It is easy to see, that 
for any two states of the population, there is a positive probability of the transition between 
them in some finite number of time steps. We have therefore obtained an irreducible Markov 
chain with n + 1 states. It has a unique stationary state (a probability mass function) which 
we denote by /x^^. For any z E Q, fJ^niz) is the frequency of visiting the state z in the long run. 
The following definition was introduced by Foster and Young [9]. 

Definition 2; e Q is stochastically stable if lime_>o Uniz) > 0. 



3 Equilibrium transitions 

We review here recent results concerning the dependence of stochastic stability of equilibria on 
the number of players. 

They are based on a certain tree representation of stationary states of irreducible Markov 
chains ([11, 12, 13]; see also Appendix). We assume that at any time period, each individual 
has a revision opportunity with a small positive probability r. It follows that z — and z — n 
are the only absorbing states. After a finite number of steps of the noise-free dynamics, we 
arrive at one of these two states and stay there forever - there are no other recurrence classes. 
Therefore to obtain a stationary state in the limit of zero noise, it is enough to count a number 
of mistakes the population needs to evolve between these states. If one requires, for example, 
fewer mistakes to evolve from z = to z = n than from z = n to z = 0, then ^ = n is 
stochastically stable. 
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Robson and Vega-Redondo proved that for a sufficiently big number of players, the efficient 
strategy A is stochastically stable [7]. They showed that hmg_,.o //^(n) = 1 which means that 
in the long run, in the limit of no mistakes, all individuals play A. 

However, their proof requires the number of players to be sufficiently big. It was showed in 
[14] that the risk-dominant strategy B is stochastically stable if the number of players is below 
(2a — c — 6)/(a — c). 

Let us recall the proof. If the population consists of only one i?-player and n — 1 A-players 
and if c > [a{n — 2) + b]/{n — 1), that is n < (2a — c — 6)/(a — c), then ttb > 'Ka- It means that 
one needs only one mistake to evolve from z — n to z — li is easy to see that two mistakes 
are necessary to evolve from z — Q to z — n which finishes the proof. 

To see stochastically stable states, we need to take the limit of the zero noise level. It was 
showed in [14] that for any arbitrarily low fixed noise level, if the number of players is big 
enough, then in the long run only a small fraction of the population plays the efficient strategy 
A. Smaller the noise level is, fewer individuals play A. 

Let us note that the above theorem concerns an ensemble of states, not an individual one. 
In the limit of the infinite number of players, that is the infinite number of states of the system, 
every single state has zero probability in the stationary state. It is an ensemble of states that 
might be stable. Ensemble and stochastic stability in spatial games with local interactions were 
recently discussed in [15, 16, 17]. For an interesting discussion on the importance of the order 
of taking different limits (r — > 0, n — > oo, and e — > 0) in evolutionary models (especially in the 
Aspiration and Imitation model) see [18]. 

Now we combine the above theorems and obtain 
Theorem 

For any 5 > and /? > there exist e((5, /9) and rii < 712 < ^3(e) < rii^e) such that 

if n< ni = ^^^z^, then //^(^ = 0) > 1 - (5, 

iin2 <n < nz{e), then ^^^{z — n) > 1 — 5, 

if n > n4(e), then //^(-z < (5n) > 1 — 5 for a sufficiently small r. 

We see that for a fixed noise level, when the number of player increases, the population 
undergoes twice a transition between its two equilibria. Of course, for any fixed number of 
players, n > n2, if the noise level is sufficiently small, then almost all indivisuals will play in 
the long run the efficient strategy A. 

In order to study the long-run behavior of stochastic population dynamics, we should esti- 
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mate the relevant parameters to be sure what hmiting procedures are appropriate in specific 
examples. Equilibrium transitions in other stochastic dynamics of finite populations were re- 
cently investigated in [19, 20]. 

Appendix 

The following tree representation of stationary distributions of Markov chains was proposed 
by Freidlin and Wentzell [11, 12], see also [13]. Let {^l,P) be an irreducible Markov chain 
with a state space Q and transition probabilities given hj P'- : Q x Q ^ [0,1]. It has a unique 
stationary distribution, also called a stationary state. For X E Q, let an X-tree be a directed 
graph on fl such that from every Y ^ X there is a unique path to X and there are no outcoming 
edges out of X. Denote by T{X) the set of all X-trees and let 

?i^)= E n ny,y'), m 

dGT{X) {Y,Y')ed 

where the product is with respect to all edges of d. We have that 
for all X en. 

Let us assume now that after a finite number of steps of the noise-free dynamics, i.e. e = 0, 
we arrive at one of two absorbing states, say X and Y, and stay there forever - there are 
no other recurrence classes. It follows from the tree representation that any state different 
from absorbing states has zero probability in the stationary distribution in the zero-noise limit. 
Consider a dynamics in which P'^{Z, W) for all Z,W e fl, is of order e™, where m is the number 
of mistakes involved to pass from Z to W or is zero. Then one has to compute the minimal 
number of mistakes, uixy, needed to make a transition from the state X to y and the number 
of mistakes, myx, to evolve from Y to X. q{X) is of order e^^C^^) and q{Y) is of order 
It follows that if mxr < myx, then Y is stochastically stable and moreover lime_^o A*^(^) — 1- 
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